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METHOD AND SYSTEM TO IDENTIFY AND CHARACTERIZE NONLINEARITIES 
IN OPTICAL COMMUNICATIONS CHANNELS 

CROSS-REFERENCE TO RELATED APPLICATION(S) 

This application claims priority from provisional applications "METHOD TO 
IDENTIFY AND CHARACTERIZE NONLINEARITIES IN OPTICAL 
COMMUNICATIONS CHANNELS" serial number 60/253,207 filed on November 27, 
2000, and from "METHOD AND SYSTEM FOR OPTICAL CHANNEL IDENTIFICATION 
AND MODELING" serial number 60/252,711 filed on November 22, 2000. Both 
provisional applications 60/253,207 and 60/252,711 are incorporated herein by 
reference as though set forth in full. 

FIELD OF THE INVENTION 

The invention relates to methods used to characterize communication channels 
and, in particular embodiments, to methods used to characterize and model nonlinear 
optical communication channels. 

BACKGROUND OF THE INVENTION 

Signal distortion in an optical channel may be caused by nonlinearities that exist 
in various components, which comprise an optical channel. Such distortion can be 
characterized by mathematical models of the nonlinear behavior of the optical 
communication channel. By using such models, compensatory measures, for example 
equalizers that compensate for distortion, which degrades the performance of the 
channel, may be designed. 
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One source of distortion is dispersion. Dispersion may be a combination of linear 
and nonlinear components. 

5 Dispersion found in optical fibers, particularly multimode fibers, may limit the 

bandwidth in optical channels. The problem of dispersion may be more acute in 
multimode fibers in general, but it may also be a problem in single mode fibers, 
especially in long runs of fiber. Single mode fibers commonly have lower dispersion 

^ than multimode fiber. The dispersion commonly found in single mode fibers may be 
chromatic dispersion or polarization mode dispersion. The dispersion that is seen in 
single mode fibers is generally a lot less per length than the dispersion found in multi- 
mode fibers. Dispersion is cumulative, however, and so tends to increase as fibers 

15 

increase in length. Accordingly, methods discussed herein in terms of multimode fibers 

may apply equally well to single mode fibers. 

In multi-mode fibers, the multiple modes of propagation within the fiber 
2Q commonly cause dispersion. Generally, in the various modes of propagation within a 

fiber, the light signal has different speeds of propagation. So if a light pulse is sent from 

a transmitter, the pulse will propagate in multiple modes traveling at different speeds. 

The pulse, traveling in each mode, will reach the receiver at a different time. The 
25 multiple times of arrival of the input pulse traveling in several modes create a distorted 

pulse, somewhat like a spread out version of the transmitted pulse. 

Optical channels are inherently nonlinear. Electromagnetic fields can be 

described by Maxwell's equations, which are linear. Because Maxwell's equations are 
^® linear, the principle of superposition applies to the electromagnetic field. What is 

modulated, however, in optical channels is not the electromagnetic field perse, it is the 

optical power. Superposition doesn't exactly apply to optical power. 

35 
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An optical photo detector is commonly a square law device. The photodetector 
responds linearly to the optical power, in fact the photo current generated by a 
5 photodetector is commonly a very accurately linear function of the optical power, but the 
power is a quadratic function of the electromagnetic field, thus creating a source of 
nonlinearity. 

It has been shown that if a laser has a large spectral width, instead of being a 

1 n 

very monochromatic laser, the significant spectral width tends to result in a linearization 
of the channel even considering the effect of the optical power and the square law 
nature of the photodetector. In other words if a laser has a very monochromatic, narrow 
spectrum then the channel tends to behave less linearly than does a laser that has a 

15 

wider spectrum. 

Nonlinear behavior in an optical channel may depend to varying extent on the 
properties of the photodetector and the fiber. Additionally there is a possibility of 

2Q distortion from the laser itself. If there is dispersion in the laser, for example because 
the laser has some bandwidth limitation itself, then nonlinearity of the laser could also 
contribute to the total nonlinearity in the optical channel. 

When the intensity of an optical signal is high, a fiber optic which transmits the 

25 optical signal itself may introduce nonlinearities. These nonlinearities may result from 
the fact that the index of refraction of the fiber optic depends, to some extent, on the 
intensity of the optical signal itself. This effect commonly gives rise to nonlinearities 
known as "four-wave mixing", "self-phase modulation", and "cross-phase modulation". 

on 

Additional nonlinearities may result from the phenomena known as stimulated Raman 
scattering and stimulated Brillouin scattering. For a more comprehensive treatment of 
nonlinearities, see the text "Fiber-Optic Communication Systems", second edition, by 

35 
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Govind P. Agrawal, John Wiley and Sons, 1997, ISBN 0-471-17540-4, which is 
incorporated by reference herein. 
5 Although the above sources of nonlinearity in an optical channel are presented 

as examples, other sources may exist. The techniques described in this disclosure may 
be applied to all kinds of nonlinearities regardless of their physical source, as will be 
clear to those skilled in the art. 

10 

SUMMARY OF EMBODIMENTS OF THE INVENTION 

In one aspect of the invention, the disclosure illustrates a method for modeling 
the behavior of a data channel. To create the model, the sequence of data input to the 

15 

data channel is examined. A portion of the sequence of data input to the data channel 
is used as an index to a channel model value. Output data from the channel is 
sampled. A numerical channel model value is compared with the sampled value and 

2Q the channel model value is adjusted based on the results of the comparison. 

In another aspect of the invention, a method for modeling the behavior of a data 
channel using Volterra Kernels is illustrated. A sequence of data is provided to the data 
channel. A part of the sequence of data input to the channel is provided to the Volterra 

25 Kernel model of the channel. The Volterra Kernel (VK) model produces a channel 
model value. The data values measured from the channel are compared to the VK 
channel model value and the results used to update the Volterra Kernel model. 

30 BRIEF DESCRIPTION OF THE DRAWINGS 

The features, aspects, and advantages of the present invention, which have 
been described in the above summary, will be better understood with regard to the 
following description, appended claims, and accompanying drawings where: 
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Figure 1 is a graphical illustration of an environment in which embodiments of the 
present invention may be used. 
5 Figure 2 is a graphical illustration of an equipment setup, which may be used to 

characterize optical channels. 

Figure 3 is a table listing of fibers, which were measured using the measurement 
setup illustrated in figure 2. 
^ Figure 4 is a graphic illustration of an example of a system that may be used for 

channel identification. 

Figure 5 is a graphical illustration of channel identification based on Volterra 
Kernels (VK). 

15 

Figure 6 is a graph illustrating measurement of a channel having an 850 
nanometer VCSEL (Vertical Cavity Surface Emitting Laser) and fiber F0, and 
comparing the signal measurements to linear and nonlinear channel models. 
2Q Figure 7 is a graph illustrating the measurement of a channel having an 850 

nanometer VCSEL and fiber F0, and comparing the amount of error generated by 
linear and nonlinear channel models. 

Figure 8 is a graph representing an impulse response and a second order 
25 Volterra Kernel for an 850 nanometer VCSEL and fiber F0. 

Figure 9 is a table of the signal to noise ratios computed for linear and nonlinear 
channel models for a channel operated at a wavelength of 850 nanometers. 

Figure 10 is a graph comparing linear model, measured signal and error within 
30 a channel using a fiber F0 at a wavelength of 1310 nanometers. 

Figure 1 1 is a graph of the impulse response and a second order Volterra Kernel 
for a channel using fiber F0 and a distributed feedback (DFB) laser with 1310 
nanometer wavelength. 

35 
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Figure 12 is a table of signal-to-noise ratios for a linear model of a channel 
employing1310 nanometer DFB laser. 
5 Figure 1 3 is a block diagram of a nonlinear optical channel equalizer, according 

to an embodiment of the invention. 

Figure 14A is a block diagram of a nonlinear optical channel equalizer that 
employs a Volterra Kernel estimator according to an embodiment of the invention. 
10 Figure 14B is a block diagram of a non-linear optical channel equalizer 

employing a lookup table channel estimator, according to an embodiment of the 
invention. 

Figure 15 is a Viterbi decoder employing a nonlinear channel estimator, 

15 

according to an embodiment of the invention. 

DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION 
2Q The listed figures describe and illustrate various aspects of the present invention. 

Accordingly, Figure 1 is a graphic illustration of an environment in which 
embodiments of the present invention may be used. The environment illustrated at 1 01 
is an information distribution system, such as may be found, for example, in a computer 
25 network accessing a remote database. 

In Figure 1, users 105 are coupled to a server 103. The users then may 
communicate with remote database 107 using fiber optic 109. Fiber optics may be 
used to communicate large amounts of data between points because of their large 
30 bandwidth. 

The data rate of optical channels is, in part, limited by nonlinearities in the 
channel. The term "channel", as used herein, includes the laser, amplifiers that drive 
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the laser, the fiber optic, the photo detector receiver and all elements included in the 
transmission and reception of a fiber optic signal. 
5 For characterizing optical channels, the measurement setup illustrated in Figure 

2 was used. 

Figure 2 is a graphical illustration of an equipment setup used to characterize 
optical channels. A signal is generated by modulating a laser 219. The modulated 

1 n 

1 u laser signal is provided to a fiber optic 215. In the embodiment illustrated in Figure 2, 
the signal provided to the optical channel was a 1 Gigahertz (GHz), 127 bit, pseudo- 
random binary sequence provided by bit error rate tester 201. The resulting received 
signal was detected by a photodetector 207, captured using an oscilloscope 205 and 

15 

stored as a file in computer 209. Using computer 209, the received signal may be 

processed and the nonlinearity that exists in the received signal can be identified. The 

approach taken is that no attempt is made to identify the source of the nonlinearity as 
2Q belonging to a specific channel component such as a photodetector or laser or any 

other source. Instead the nonlinearity in the received signal was examined without 

being concerned for the physical source of the nonlinearity. 

Two types of lasers were tested. The first type was a 850 nanometer VCSEL 
25 (vertical cavity surface emitting laser) and the second type of laser was a 1310 

nanometer DFB (distributed feedback) laser. All measurements were done at a data 

rate of 1 gigabit per second. 

The bit error rate data tester 201 generated a pseudo random binary sequence 

on 

ou that was periodically repeated. The channel measurements were performed at a data 
rate of 1 gigabit per second, but the method may be applied to any data rate. 

In the setup illustrated in Figure 2 free space optics and an optical bench 221 
were used. The laser was aligned using alignment screws so that the laser 219, the 

35 
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fiber 215 and the lens were arranged to focus the light into fiber 215. Laser 219 
requires a DC bias current provided by bias power supply 203. A bias Tee 21 1 allows 
5 the combination of the data signal with a bias in order to apply a correctly biased data 
signal to the laser 219. 

A length of fiber 215 is contained in a spool so that lengthy fibers up to more 
than 1 kilometer may be used. In the case of single-mode fibers, the length of the spool 
^ could exceed 700 km. Photodetector 209 is a commercial unit having a bandwidth of 
1 0 gigahertz. The oscilloscope 205 is a real time sampling oscilloscope which samples 
at 8 gigahertz. Because the data rate used was 1 gigabit per second, the received 
signal was sampled on average 8 times per bit. Samples of the received signal were 
captured in the memory of the oscilloscope and then downloaded to the computer 209 
in the form of a file. Typical received signal data files contained about 65,000 data 
points representing received samples of the pseudo random input sequences of 127 
bits. 

Once the data points have been accumulated they may be processed and 
information about the behavior of the optical channel, particularly the nonlinearity, may 
be extracted. Additionally any possible nonstationarity of the channel may be identified. 

25 In other words, if the channel changes in time, such nonstationarity may be detected. 
In the measurements taken, however, no significant nonstationarity was observed. 

Figure 3 is a table listing of fibers, which were measured using the measurement 
setup illustrated in figure 2. Six different fibers, having core diameters from 50 to 62-1/2 

30 microns and the lengths from 270 meters to about 2.2 kilometers, were tested. All of 
the fibers exhibit some dispersion phenomenon and bandwidth limitation to varying 
extent. 



35 
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To model channel nonlinearity two different mathematical methods, representing 
nonlinear functions having memory, were chosen. Because dispersion is inherently 

5 a function having memory, the resulting channel nonlinearity is a nonlinearfunction with 
memory, as opposed to memoryless nonlinearity. Memoryless nonlinearity may be 
represented by a function of the form Y = F(X), with the nonlinearity an arbitrary 
nonlinear function F of a single variable. Nonlinearity with memory can be described 

^ as a nonlinearfunction of a number of consecutive samples of the transmitted data, so 
that the nonlinear function with memory is of the form Y = F( X^X^X^,..). In other 
words nonlinearity with memory is a function of several variables, where the variables 
are exemplarily samples of the transmitted bit sequence. Nonlinearity with memory is 

15 

a more general kind of nonlinearity that may be found in optical channels. In other 
words the characteristic of an optical channel is not only a function of the current bit, but 
depends also on previously transmitted bits. 

2Q The nonlinearity in an optical channel can be modeled as a look-up table, for 

example as a nonlinear function of the five most recently transmitted bits. In a digital 
transmission system the transmitted bits have 2 levels, 0 and 1 (or +1 and -1 depending 
on the convention that is used). Because of the discrete nature of the transmitted bits, 

25 the function characterizing a channel depending on the five most recent bits can only 
have 2 5 =32 values. Accordingly there are 32 possible values that model the channel 
by listing all the values for all the possible sequences of the most recent 5 input bits. 
Such a model can be implemented as a look-up table. The look up table is one model 

^ with memory for characterizing an optical channel having nonlinearity. 

A second optical channel model is based on a Volterra series. Volterra series 
have been studied in mathematics for a considerable time. Volterra series are a way 
to represent nonlinear functions. In general Volterra series are infinite series, i.e. a 

35 
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series with infinite number of terms. In the present case, however, since the transmitted 
data is binary, the number of terms is finite. The number of terms in the series used to 

5 model a binary optical channel is 2 to the power N, where N is the number of most 
recent transmitted bits, which are taken into account. That is the same number of terms 
as the number of entries in a look-up table. Accordingly, no matter what description is 
used, whether the look-up table approach or the Volterra series approach, the number 

1 0 of required elements in each model is 2 to the power N, where N is the number of most 
recent bits transmitted that are taken into account. In the look-up table approach the 
2 to the N parameters are the values of the channel for all the possible values of the 
most recent N input bits. 

15 

For the Volterra series, the parameters are the coefficients of the series and 
because the series has 2 N terms there are 2 N coefficients. 2 N is the same number of 
parameters as required by the look up table model. Accordingly regardless of which 

2Q description is chosen, the maximum number of parameters that is needed to completely 
specify the nonlinear function is the same, i.e. 2 N . It can be shown that the Volterra 
Kernel representation is mathematically equivalent to the look-up table. In fact a 
mathematical transformation known as the Hadamard transformation can transform the 

25 look up table model to the Volterra Kernel model and Volterra Kernel model to the look 
up table model. 

N is generally an empirically determined value, the exemplary value of N=5 in the 
present embodiment appears to work very well. The value of N, of course could 

on 

depend on the fibers, wavelength of the laser and the rate of the data being transmitted. 
In principle it is possible to find fibers where N is larger or smaller, but in present 
embodiments N=5 is sufficient. 

35 
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An advantage of choosing the Volterra series description is that there is a way 
to make the description somewhat compact by grouping terms. By grouping the terms 
5 2 N terms are transformed into 2 (N1) convolution-like terms having a shift register-type 
sequence. That means, for example, the terms that go into the sequence of grouped 
terms are products of bits. So, for example, a X N term would be multiplied by an 
term and so on. The shift register sequence of grouped terms means shifted versions 

1 n 

u of the product. So, for example, this shift register sequence would be in the form X N * 
X^, X n-1 * X N _ 2 , X N _ 2 * X N _ 3 and so on. This is shift register-type sequence is one that 
can describe terms of the Volterra series as a convolution of this kind of shift register- 
type sequence with a certain kernel. A kernel is an element that resembles an impulse 

15 

response. A kernel is not an impulse response in the traditional sense, but a traditional 
impulse response is used as a linear portion of the nonlinear kernel function. 

Terms involving a convolution of a shift register type sequence with a certain 

2Q kernel, are referred to as Volterra Kernels. There are multiple Volterra Kernels 
corresponding to different kinds of products of bits. The maximum number of Volterra 
Kernels needed to represent an N bit sequence to is 2 N ~ 1 . So, for example, in the case 
where N=5, the total number of terms in the Volterra series is 2 N or 32 terms. But these 

25 32 terms can be mapped into 1 6 Volterra Kernels. Sixteen Volterra Kernels suffice and 
the number of Volterra Kernels is equal to 2 (N_1) , or in the present example 2 4 or 1 6. So 
there are 16 Volterra Kernels and the Volterra Kernels may provide a more convenient 
and more compact description of a nonlinearity particularly because commonly in many 

30 applications the number of Volterra Kernels is actually less than 2 (N1) . Although the 
number of Volterra Kernels to theoretically define a Volterra Kernel model is 2 {N_1) , some 
of the kernels are very small and can be neglected. In some situations the channel may 
be effectively modeled by one or two Volterra Kernels. That is the situation in the 

35 
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optical channels of the present embodiment. In present embodiments the nonlinearity 
is simple enough so that it can be described completely by a linear term and a single 
5 Volterra Kernel, indicating that a fairly simple kind of nonlinearity is present. 

Because the nonlinearity is simple doesn't mean that its negligible. In fact in 
present embodiments the nonlinearity, though simple, is a very strong. Because the 
nonlinearity has a simple description, simple equalizers may be built. In other words 

1 n 

1 u nonlinear equalizers with memory which are relatively simple but very powerful may be 
built. Such equalizers can be very effective because they can deal with the nonlinearity 
that is strong whereas a traditional kind of linear equalizer might fail completely. 

Figure 4 is a graphic illustration of an example of a system that may be used for 

15 

channel identification. The system is implemented in software. 

Input bits 401 are provided to a channel 403. What is used for the channel 403 
is the measurements that were taken with the setup illustrated in Figure 2. So the 
2Q channel input and channel response are actual measured values. 

In the present embodiment the measurements are stored in a file and then 
processed off-line in a computer using a software program that implements the 
remainder of the system illustrated in Figure 4. This was done for the sake of 
25 convenience because the equipment for the setup in Figure 2 was readily available. 
There is however no constraint to prevent the present disclosed methods from being 
implemented as part of a real time or embedded system. 

The look-up table 409 is adaptive. That is the value in it can be adapted using 

Of] 

ou the LMS (least-means-squared) algorithm. Basically what is done is to compare the 
measured sequence from the channel 403 (the file of received values stored by 
computer 209) with the output of the look-up table. The addresses in the look up table 
409 are received from a shift register 407 representing the values of the latest 5 input 

35 
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bits provided as input to the channel. Initially the look-up table values are initialized to 
all zeros started from situations where there is no knowledge of the nonlinearity. In 
5 such a case the error will be equal to the input signal. So the error will be large initially. 
The values in the look up table can be adjusted using an LMS algorithm to update the 
look-up table according to the error 405 observed. The LMS algorithm will converge so 
that it tends to drive the error to zero or as close as possible to zero. 

1 o 

Once the look-up table converges the error will be small or zero. In actual 
practice the error may not be exactly zero because the input signal may include some 
random noise. Additionally it is also possible the look-up table doesn't perfectly model 
all the nonlinearity. There may be some residual unaccounted for nonlinearity because 

15 

N was chosen equal to 5 but in practice a N of 8 or 1 0 may have been required to more 
closely model the fiber. The choice of N is an implementation detail depending on the 
type of fiber, data rate, and a variety of other factors. 

2Q Additionally in figure 4 the look up table is like having an array where the address 

is created by looking at 5 consecutive bits, but the number is accessed is a single 
number that could be represented by multiple bits which could be 1 6 bits or 24 bits, and 
as such may contain some quantization error. 

25 Figure 5 is a graphical example of channel identification based on Volterra 

Kernels. The Volterra Kernel approach is an approach mathematically equivalent to the 
use of a look-up table, as illustrated in Figure 4. In Figure 5, four Volterra Kernels are 
used. Normally there would be 16 Volterra Kernels present in a case where N is equal 

Of] 

to 5. In some cases some of the Volterra Kernels may be negligible and depending on 
the Bit Error Rate (BER) required only the most important ones may need to be taken 
into account in the model, as for instance illustrated in Figure 5. For example in Figure 
5 there is a linear term which is represented by the Kernel labeled FIR 511 "FIR" 

35 
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indicates that each Kernel is represented by a finite impulse response (FIR) filter. In 
Figure 5, FIR 51 1 directly takes a sequence of input bits and creates a convolution of 
5 the input sequence with an impulse response of FIR filter 511. Hence FIR 511 is a 
linear term, i.e. a traditional convolution. So if only a term represented by FIR 511 is 
needed for the channel model, the channel is essentially linear. If, in addition to FIR 
511, other terms are present then the model is nonlinear. 

10 

FIR 51 3 is a Volterra Kernel which is associated with products of two consecutive 
bits, i.e. a second order Volterra Kernel. FIR 513 is a second order Volterra Kernel 
because it accepts a term 505 which is a product of the most recent input bit and the 
bit preceding the most recent input bit. The most recent input bit is designated as A N 

15 

and the bit preceding the most recent bit is designated A^. 

Second order Volterra Kernel FIR 513 is associated with two (consecutive) bits 
i.e. bits that are one time delay apart, it accepts the product of two (consecutive) bits. 

20 Another second order Volterra Kernel 515 is produced by taking the product of bits that 
are two time delays apart, 501 and 503 representing time delays. The most recent bit 
and the bit prior to the second most recent bit are multiplied and result in output 507, 
which is the accepted by FIR 515. In other words A N is multiplied byA N _ 2 , here, because 

25 this is delayed by 2 units. FIR 51 5 is a Finite Impulse Response filter (just as FIR 51 1 
and FIR 513) but the input, instead of being bits is the product of 2 bits, and so it is a 
second order Volterra Kernel. FIR 515 accepts the product of bits A N and A N _ 2 , so it is 
a different type of Volterra Kernel than FIR 513 that accepts sequential bits. Both FIR 

on 

JU 515 and FIR 513 are second order. 

A third order Volterra Kernel FIR 517 is determined by the product (509) of 3 bits. 
509 is a product of 3 consecutive bits A N , A NO and A N . 2 . Accordingly FIR 517 is a third 
order Volterra Kernel. 

35 
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In the simplified mode! illustrated in Figure 5 only the aforementioned 4 kernels, 
the linear kernel, two second orders kernels, and a third order kernel are present. In 
5 practice, if a nonlinearity were very strong, the Volterra Kernel model could have 16 
kernels or more if N g reater than 5 (where N is the number of most recently transmitted 
bits to be considered) . In the present embodiment out of the 16 Volterra Kernels only 
the four illustrated in Fig. 5 are significant, the others are negligible. 

1 n 

In the present embodiment FIR 51 1 and FIR 513 are much more significant than 
FIR 515 and FIR 517, and in fact FIR 515 and FIR 517 could be completely neglected. 

The Volterra Kernel model tends to simplify the description of the channel 
because it reduces the number of independent terms that need to be taken into 

15 

account. Volterra Kernels may group together all the terms that are time-shifted 
versions of each other. In the more general Volterra Series description, all the terms 
are represented independently. When groups of terms of a Volterra Series are time- 

2Q shifted versions of each other, they may be associated with each other, and all the ones 
that are time-shifted versions of each other may be grouped as a single Volterra Kernel. 

The two equivalent representations - the look-up table model and the Volterra 
Kernel model are mathematically equivalent because the Hadamard transformation can 

25 transform one into the other. The reason to choose one or the other is essentially ease 
of implementation. The model that is simpler is commonly dependent on the particular 
problem that is trying to be solved. For example, if a strongly nonlinear function with 
short memory is present (meaning small amounts of dispersion) the look-up table 
probably provides a simpler model. The look up table is likely simpler because, if the 
dispersion is not large N will not be large. If N is not large then 2 N is not a very large 
number of coefficients. But if the memory span of the nonlinearity (i.e. N) grows then 
the look-up table grows exponentially because the size of the look-up table is 2 N . The 

35 
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size of the look up table may become prohibitively complex for large N. In some cases 
Volterra Kernels may provide a simpler model when N is large and the nonlinearity has 
5 a simple form which can be described with few kernels, such as the previous example 
that could be limited to two kernels. 

If N is large, it will generally indicate that there is a lot of dispersion in the 
channel, but the form of the nonlinear function could be very simple. The function may 
be simple enough that it can be described, as in the previous example, by only two 
Volterra Kernels. If N is large and the Volterra model is simpler, the nonlinearity may 
be completely modeled with two kernels, even though N is very large. Accordingly in 
some cases the Volterra Kernel description is simpler. In other cases the look-up table 

15 

description is simpler. 

In the studies performed, the amount of dispersion necessitated an N on the 

order of 5, but in principle, for severely bandwidth limited fibers, N could be larger. In 
2Q the case where two Volterra Kernels are sufficient to create an accurate channel model, 

regardless of how large N is, the Volterra Kernel description is generally simpler. 

Figure 6 is a graph illustrating channel measurement, of a channel having an 850 

nanometer VCSEL laser and fiber F0, and comparing the signal measurement with 
25 linear and nonlinear fiber models. In Figure 6 the solid line corresponds to the 

observations, the dotted line represents the linear model, and the dashed and dotted 

represents the nonlinear model. As can be seen from 601 , 603 and 605 the nonlinear 

model approximates the measurements of the channel to a greater degree than the 

on 

linear model. Accordingly if the channel model were limited to a linear model a good 
fit for the transmitted data would not be achieved. In contrast using the nonlinear model 
a significantly closer fit may be obtained. 

35 
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Figure 7 is a graph illustrating channel measurement of a channel having an 850 
nanometer VCSEL laser and fiber F0, and comparing the signal measurement with 
5 amount of error generated by linear and nonlinear channel models. Figure 7 is a graph 
representing the same time as the graph of Figure 6 except that the errors instead of 
the signals themselves are shown. In Figure 7 the linear model error is represented by 
the lines with circles and the nonlinear model is represented by the dotted line. As can 

1 n 

be readily appreciated, by observing the nonlinear model points at 701 A, B, and C and 
comparing it with the linear model points at 703 A, B, and C the nonlinear model 
achieves a significantly smaller error. 

Figure 8 is a graph representing the impulse response and a second order 

15 

Volterra Kernel for a channel having an 850 nanometer wavelength VCSEL laser and 
utilizing fiber F0. Figure 8 illustrates an impulse response (the linear term of the overall 
response) at 801. Figure 8 also illustrates the response of the second order Volterra 

2Q Kernel at 803. in the present embodiment, the second order Volterra Kernel, which is 
associated with products of two consecutive bits, is the only significant nonlinear kernel. 
So the response of the second order Volterra Kernel generated from the product A N x 
A^, i.e. FIR 513 in figure 5. FIR 511 corresponds to the linear kernel response 801 

25 and FIR 513 corresponds to the second order response 803. Using just the Volterra 
Kernels of FIR 511 and FIR 513 a close approximation of the channel which includes 
nonlinear elements and memory was achieved. The same method was used to 
examine the remaining fibers in the table of Figure 3. The signal-to-noise ratio for the 
linear model and for the nonlinear model (which was a linear model with nonlinear 
addition) was computed for each fiber. 

Signal-to-noise ratio is defined as 10 times the base 10 log of the signal power 
divided by the error power. To compute the signal-to-noise ratio the power for the 

35 
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signal was computed. The sum of the squares of the samples of the signal yields the 
total power of the signal and the sum of the squares of the samples of the error yields 
5 the error power. 

There are two versions of the error power, one of them for the linear model 
containing one Volterra Kernel and the other for the nonlinear model containing the 
additional nonlinear Volterra Kernels. The result of the error computations for the linear 

1 n 

and the nonlinear model is illustrated in the table of figure 9. As can be seen from the 
table of figure 9 the signal-to-noise ratio for the linear model is in the range of 12 to 13 
dB. In the nonlinear model the error is significantly smaller, and correspondingly the 
signal to noise ratio is significantly higher in the range of 23 to 25 dB. The nonlinear 

15 

model yields a model having approximately a 10 to 13 dB better signal to noise ratio, 
which is a significant improvement. 

The previously described measurements were all done at 850 nanometers 

2Q wavelength. In the measurements taken, the nonlinearity was much larger at 850 
nanometers than at 1310 nanometers. If the same measurements are repeated at 
1310 nanometers, the linear model appears to be very adequate. 

Figure 1 0 is a graph of measured signal vs. linear model and the error between 

25 the measured signal and the linear model. Accordingly with the present data rate and 
fibers the linear channel model at 1 31 0 nanometers appears to be fairly accurate. The 
graph of figure 10 illustrates the measured signal as a solid line. The graph of figure 
1 0 is the result of a linear model shown as the line with circles. The line with diamonds 

on 

represents the error. 

Figure 1 1 is a graph of the impulse response and a second order Volterra Kernel 
for the channel comprising fiber F0 and a DFB laser having a 1310 nano-meter 
wavelength. The impulse response is illustrated at 1 101 and the second order Volterra 
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Kernel is illustrated at 11 03. The second order Volterra Kernel is almost zero and most 
likely would not have to be taken into account in a model of the channel. 
5 Figure 1 2 is a table of signal-to-noise ratios for a linear channel model having a 

DFB laser operating at a wavelength of 1 31 0 nanometers. All of the channel signal to 
noise ratios of all measured channels operating at a wavelength of 1310 nanometers 
were in the range of 26 to 28 dB indicating that a nonlinear term may not be necessary 

10 

at a wavelength of 1310 nanometers, for a one gigahertz data rate. 

Additionally measurements were done including cases where mechanical motion 
of a fiber was introduced during the measurement process to examine motion induced 
nonstationarity of the general response. These observations for nonstationarity were 

15 

done because there is some belief that the optical channel is not fixed in time and if 
vibration is introduced in the fiber some redistribution of the modes in the fiber may 
change the response of the fiber and accordingly the channel may be time-dependent. 
2Q No evidence of any significant change in the fiber response as a result of motion was 
observed. 

Slow changes in the channel may be compensated by the LMS algorithm 
adjusting the channel model to track those types of changes. 
25 A program listing of a computer program used for the channel studies is included 

as Appendix A. Essentially, the program takes an input block of data from the 
measurements processes it, and then trains the canceller. It then computes the 
Volterra Kernels coefficients of the model using the linear model or the nonlinear model. 

on 

The program computes a signal to noise ratio and it prints out the results. 

Figure 13 is a block diagram of an optical receiver including a nonlinear optical 
channel equalizer. The nonlinear optical channel equalizer 1 300 accepts a signal which 
includes linear and nonlinear components. 
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The electrical signal commonly results from converting han optical signal to the 
electrical domain using an optical to electrical converter 1307, such as for example a 
5 PIN diode, avalanche photodetector (APD), or another detector. The optical to 
electrical converter signal is amplified using atransimpedance amplifier (TIA) 1309 and 
a postamplifier 1311. The postamplifier 1311 generally should not hard-limit a signal 
provided to it as hard limiting may introduce nonlinearity. 

1 0 

After processing the signal provided to it the postamplifier 131 1 may provide a 
processed signal to an optional analog processing block 1313. The analog processing 
block 1 31 3 may provide such analog processing as filtering and automatic gain control, 
prior to providing an output signal to an analog to digital converter 1315. The output 

15 

signal from the analog to digital converter converts the electrical signal provided by the 
optional analog processing block 1313 to a multibit digital representation, prior to 
providing the multibit digital representation to a nonlinear optical channel equalizer 

2Q 1300. The nonlinear optical equalizer 1300 operates in the digital domain, and maybe 
implemented using special-purpose digital logic or a general purpose programmable 
digital signal processor. Prior to entering the nonlinear optical equalizer 1300 the 
multibit signal may undergo additional preprocessing steps such as filtering, linear 

25 equalization, interference cancellation, etc. in an optional digital processing block 1317. 
Equalizer 1300 is a decision feedback equalizer comprising a detector such as slicer 
1303, a compensator such as subtractor 1301 and a feedback block 1305. The 
feedback block 1 305 includes a nonlinear channel estimator employing a lookup table 

30 

as illustrated in Figure 4 or the Volterra Kernel approach as illustrated in Figure 5. A 
difficulty with using the look up table approach of Figure 4 or the Volterra Kernel as 
illustrated in Figure 5 is that in both cases the Volterra Kernel or look up table 
coefficients need to be identified. One solution to the problem of coefficient 
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identification is to have the transmitter send a training sequence, which is known a 
priori, to the receiver. The nonlinear channel estimator 1 305 can then be trained using 
5 the known training sequence. In such a case, the functioning of the nonlinear channel 
estimator 1 305 would proceed as in Figure 4 or Figure 5 as the bit sequence would be 
known and differences produced by the channel could be readily identified. 

In practice, a training sequence may not be required because the decisions 1 307 

1 o 

of the decision feedback equalizer 1 300 and the nonlinear channel estimator 1 305 may 
still converge. Initially, because the channel estimator 1 305 is not trained, the decisions 
1307 will have multiple errors. There is however, in most channels, a bias towards 
good decisions. Most decision feedback equalizers will converge if the channels do not 

15 

exhibit excessive intersymbol interference (ISI). Accordingly, if the coefficients in the 
nonlinear channel estimator 1305 are incorrect initially, there are both good and bad 
decisions. As long as the good decisions outnumber the bad decisions, there will be 
2Q a converging trend and the nonlinear channel estimator 1305 will eventually tend to 
converge. 

The nonlinear channel estimator 1305 may also be trained using an LMS 
algorithm as illustrated in Figure 4 or 5. As the coefficients begin improving in the 
25 nonlinear channel estimator 1 305, the improvement helps make a larger percentage of 
correct decisions, which further assists in the convergence of the nonlinear channel 
estimating filter 1305. Once the values in the estimating filter converge, the nonlinear 
channel estimator 1 305 will have modified the coefficients so that essentially all correct 

on 

decisions are made. Any incorrect decisions made by the system tend to corrupt the 
values in the nonlinear channel estimator 1305, however, as long as the correct 
decisions out number the incorrect decisions the nonlinear channel estimator will 
converge to an essentially correct solution. In other words, as long as the majority of 
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decisions are correct the coefficients of the nonlinear channel estimator 1 305 will have 
a net movement in the correct direction and converge. 

Once there is an initial convergence in the nonlinear channel estimator 1 305 the 
signal to the slicer 1303 is improved because at least some of the intersymbol 
interference is cancelled in summation unit 1301. As a result the number of correct 
decisions 1307 increases which leads to a better channel estimation. Past decisions 
are stored within the nonlinear channel estimator 1305 and are used to compute a 
replica of the intersymbol interference in order to provide it to summation unit 1301 . 

If a Volterra Kernel structure similar to Figure 5 is used for the nonlinear channel 
estimator then the memory of past decisions are stored in the FIR filters 511 through 
517, which each nominal contain a FIR delay line structure. If the look up table 
approach is used then the memory for the past decisions are stored within a shift 
register, such as that illustrated in 407 at Figure 4. 

Figure 14A is a block diagram of an exemplary nonlinear channel equalizer as 
may be used in the nonlinear decision feedback equalizer of Figure 1 3. The nonlinear 
channel estimator Figure 14 is similar to the Volterra Kernel model illustrated in Figure 
5. The Volterra Kernel model of Figure 5 was used as an illustration of characterization 
of the channel. The nonlinear channel estimator illustrated in Figure 14 has an LMS 
update loop similar to Figure 5. Additionally, similarto Figure 5, the number of Volterra 
Kernels employed would depend upon the characteristics of the channel and the laser 
used to drive the channel. 

In Figure 14A, a signal 1427, which has been provided from an optical channel 
and converted to an electrical signal, is accepted by an optical channel equalizer, 
shown generally at 1400. The equalizer 1400 accepts the signal 1427 into a summation 
unit 1301 where a (negated) correction signal 1429, representing the predicted inter- 
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symbol interference, is added. Prior to the training and convergence of the Volterra 
Kernel estimator correction signal is zero or an arbitrary value. Slicer 1303 makes 
5 decisions 1307 as to the values of the signal transmitted. Decisions 1307 are accepted 
by Volterra Kernel estimator 1401 , which models the nonlinearities of the channel that 
provides the signal to the equalizer 1400. The output of the Volterra Kernel estimator 
provides a value of predicted intersymbol interference 1429, which can then be 
subtracted from the accepted signal in order to accurately recreate a transmitted signal 
which has been transmitted. 

The error in the predicted intersymbol interference 1429 is estimated by 
comparing decisions 1307 from slicer 1303 with the input signal 1422, which has been 

15 

adjusted in summation unit 1 301 by removing the predicted intersymbol interference ISI 
1429. The decisions 1 307 and error 1425 are used to adjust the FIR filters 51 1,513, 
515, and 517, which exemplarily illustrate the FIR filters which comprise the Volterra 
2Q Kernel estimator 1 401 . In such a manner, the Volterra Kernel estimator may be trained 
and compensate for non-linearities within the channel that provides the signal to the 
equalizer 1400. 

Figure 14B is an exemplary block diagram of a non-linear optical channel 
25 equalizer employing a lookup table channel estimator, according to the embodiment of 
the invention. In Figure 14 B an LMS algorithm may be employed in adjusting the 
values within the look up table 1435. The system 14B may be operated so that training 
of the equalizer is accomplished by processing of a signal provided from an optical 

on 

channel, determining error signals and adjusting the equalizer 1401 . In such a manner 
the equalizer may be trained to model the channel parameters. Those skilled in the art 
will recognize that the equalizers may be trained by employing a known sequence. 
Because such a sequence is known a priori the difference between the response of the 
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channel and the desired response of the channel, i.e., the known sequence, can be 

readily determined. Such sequences may shorten the time necessary to train a 

5 nonlinear optical channel equalizer. 

A nonlinear channel estimator could also be built based on the look up table 

approach illustrated in Figure 4. The implementation of such a look up table version is 

a straight-forward adaptation of the channel characterization circuit illustrated in Figure 
10 4 

Figure 15 is a block diagram of a nonlinear channel estimator combined with a 
Viterbi decoder. Instead of a decision feedback equalizer a Viterbi decoder may be 
used. Nonlinear channel estimator 1 501 would be used to compute the expected signal 
going to the branch metric computations 1503. The remainder of the Viterbi decoder 
system including the actual Viterbi decoder itself, block 1505, is similar to the linear 
case. The nonlinear channel estimator 1501 is creating a model of the received signal. 

2Q If the nonlinear channel estimator 1501 has a memory comprising k bits, that is it 
remembers the last k inputs, then the Viterbi decoder 1505 would have 2 (k1) states. 
Therefore, if k=3 then the Viterbi decoder 1505 would be a 4 state Viterbi decoder. If 
the nonlinear channel estimator 1501 has a memory of 3 bits then it can support eight 

25 different expected values of the received signal. Which would be implemented with a 
look up table have eight locations. The expected values of the received signal are used 
by the branch metric computations 1 503 to compute all the branch metrics in the Viterbi 
decoder. 

30 

The Viterbi decoder illustrated in Figure 1 5 is the same as a standard type Viterbi 
decoder except for the fact that the channel estimator function is a nonlinear channel 
estimator with memory. The nonlinearity is expressed either in the form of a look up 
table or a Volterra Kernel. 
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The Viterbi decoder includes a receiver for receiving a signal including linear and 
non linear components. A non linear channel estimator computes the expected values 

5 of the received signal. A branch metrics computer computes the branch metrics based 
on the expected values of the received signal. A standard Viterbi decoder accepts the 
computed branch metrics and decodes the received signal. The non linear channel 
estimator can be fabricated using a Volterra kernel estimator or a lookup table estimator 

10 as illustrated at 1401 as illustrated at 1433 in Figure 14A or a look up table 
implementation as illustrated at 1433 in Figure 14B. 
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